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Abstract 

We consider the ergodicity and consensus problem for a discrete-time linear dynamic 
model driven by random stochastic matrices, which is equivalent to studying these concepts 
for the product of such matrices. Our focus is on the model where the random matrices have 
independent but time- variant distribution. We introduce a new phenomenon, the infinite flow, 
and we study its fundamental properties and relations with the ergodicity and consensus. The 
central result is the infinite flow theorem establishing the equivalence between the infinite flow 
and the ergodicity for a class of independent random models, where the matrices in the model 
have a common steady state in expectation and a feedback property. For such models, this 
result demonstrates that the expected infinite flow is both necessary and sufficient for the 
ergodicity. The result is providing a deterministic characterization of the ergodicity, which 
can be used for studying the consensus and average consensus over random graphs. 

Keywords: Ergodicity, random consensus, linear random model, product of random matrices, 
infinite flow. 

1 Introduction 

There is evidence of a growing number of applications in decentralized control of networked agents, 
as well as social and other networks where the consensus is used as a mechanism for decentralized 
coordination of agent actions. The focus of this paper is on a canonical consensus problem for 
a linear discrete-time dynamic system driven by a general model of random matrices, where the 
matrices are row-stochastic. Investigating whether the model reaches a consensus or not is often 
done by exploring the conditions that ensure the ergodicity, which in turn always guarantees the 
consensus. 

In this paper, we propose an alternative approach by introducing a concept of the model with 
infinite flow property, which can be interpreted as infinite information flow over time between a 
group of agents and the other agents in the network. We show that the infinite flow property is 
closely related to the ergodicity and, hence, to the consensus. In particular, we show the equivalence 
between the infinite flow and the ergodicity for a class of independent random models. 

We start by a comprehensive study of the fundamental relations and properties of the ergodic- 
ity, consensus and infinite flow for general random models, independent models and independent 

*Department of Industrial and Enterprise Systems Engineering, University of Illinois, Urbana, IL 61801, Email: 
{touril, angelia} @illinois.edu. This research is supported by the National Science Foundation under CAREER 
grant CMMI 07-42538. 



identically distributed (i.i.d.) random models. We then investigate the random models with a 
feedback property and the models with a common steady state for the expected matrices. Both of 
these properties have been used in the analysis of consensus models, but a deeper understanding 
of their roles has not been observed. We classify feedback property in three basic types from weak 
to strong and show some relations for them. Then, we study the models with a common steady 
state in expectation. By putting all the pieces together, we show that the ergodicity of the model 
is equivalent to the infinite flow property for a class of independent random models with feedback 
property and a common steady state in expectation, as given in infinite flow theorem (Theorem [7]). 
The infinite flow theorem also establishes the equivalence between the infinite flow properties of the 
model and the expected model. Furthermore, the theorem also shows the equivalence between the 
ergodicity of the model and the ergodicity of the expected model. As such, the theorem provides 
a novel deterministic characterization of the ergodicity, thus rendering another tool for studying 
the consensus over random networks and convergence of random consensus algorithms. 

The main contributions of this paper include: 1) the equivalence of the ergodicity of the model 
and the expected model for a class of independent random models with a feedback property and 
a common steady state in expectation; 2) the new insights and understanding of the ergodicity 
and consensus events over random networks brought to light through a new phenomena of infinite 
flow event, which to the best of our knowledge has not been known prior to this work; 3) novel 
comprehensive study of the fundamental properties of the consensus and ergodicity events for 
general class of independent random models; 4) new insights into the role of feedback property 
and the role of a common steady state in expectation for the ergodicity and consensus. 

The study of the random product of stochastic matrices dates back to the early work in p] 
where the convergence of the product of i.i.d. random stochastic matrices was studied using the 
algebraic and topological structures of the set of stochastic matrices. This work was further 
extended in [21 El E] by using results from ergodic theory of stationary processes and their algebraic 
properties. In [5J, the ergodicity and consensus of the product of i.i.d. random stochastic matrices 
was studied using tools from linear algebra and probability theory, and a necessary and sufficient 
condition for the ergodicity was established for a class of i.i.d. models. Independently, the same 
problem was tackled in |6J, where an exponential convergence bound was established. Recently, 
the work in [5J was extended to ergodic stationary processes in [7]. 

In all of the works [H [21 El El El El E] , the underlying random models are assumed to be either 
i.i.d. or stationary processes, both of which imply time-invariant distribution on the random model. 
Unlike these works, our work in this paper is focused on the independent random models with 
time-variant distributions. Furthermore, we study ergodicity and consensus for such models using 
martingale and super martingale convergence results combined with the basic tools from probability 
theory. Our work is also related to the consensus over random networks [S], optimization over 
random networks [2] , and the consensus over a network with random link failures [TU] . Related are 
also gossip and broadcast-gossip schemes giving rise to a random consensus over a given connected 
bi-directional communication network [HI [121 H31 EE]- On a broader basis, the paper is related to 
the literature on the consensus over networks with noisy links [151 US El HH] an d the deterministic 
consensus in decentralized systems models [121 [201 1211 [221 123] , [211 [2H I2S1 12Z] including the effects 
of quantization and delay [2H1 1221 GEH ED E21 E] • 

The paper is organized as follows. In Section |2j we describe a discrete-time random linear 
dynamic system of our interest, and introduce the ergodicity, consensus and infinite flow events. 
In Section [3j we explore the relations among these events and establish their 0-1 law and other 
properties by considering general, independent and i.i.d. random models. In Section [4j we discuss 
models with feedback properties and provide classification of such properties with insights into 
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their relations. We also consider independent random model with a common steady state in 
expectation. In Section [5j we focus on independent random models with infinite flow property. We 
establish necessary and sufficient conditions for ergodicity, and briefly discuss some implications 
of these conditions. We conclude in Section [6] 

Notation and Basic Terminology. We view all vectors as columns. For a vector x, we write Xi 
to denote its ith entry, and we write x > (x > 0) to denote that all its entries are nonnegative 
(positive). We use x T to denote the transpose of a vector x. We write ||x|| to denote the standard 
Euclidean vector norm i.e., \\x\\ = \f^~x\- We use to denote the vector with the ith entry 
equal to 1 and all other entries equal to 0, and use e to denote the vector with all entries equal 
to 1. A vector a is stochastic when a > and = 1. We write {x(k)} or {x(k)} k > to denote a 

sequence x(0), x(l), ... of some elements, and we write {x{k)} k > t to denote the truncated sequence 
x(t), x(t + 1), . . . for t > 0. For a set C and a subset 5* of C, we write S C C to denote that 5* is a 
proper subset of C. A set 5* C C such that S ^ is referred to as a nontrivial subset of C. We 
write [m] to denote the integer set {1, . . . , m}. For a set 5* C [m], we let S denote the complement 
set of S with respect to [m], i.e., 5* = {i G [m] \ i ^ S}. 

We denote the identity matrix by /. For a vector v, we use diag{v) to denote the diagonal 
matrix with diagonal entries being the components Vi of the vector v. For a matrix W, we write 
Wij to denote its (i, j)th entry, W l to denote its ith column vector, and W T to denote its transpose. 
For an m x m matrix W, we use J2i<j to denote the summation of the entries over all 
i,j G [m] with % < j. A matrix W is row-stochastic when its entries are nonnegative and We = e. 
Since we deal exclusively with row-stochastic matrices, we will refer to such matrices simply as 
stochastic. We let S> m denote the set ofmxm stochastic matrices. A matrix W is doubly stochastic 
when both W and W T are stochastic. 

We write E[X] to denote the expected value of a random variable X. We use Pr(^) and 1 ^ to 
denote the probability and the characteristic function of an event respectively. If Pr(^/) = 1, 
we say that srf happens almost surely. We often abbreviate "almost surely" by a.s. 

2 Problem Formulation and Terminology 

Throughout this article, we deal exclusively with the matrices in the set § m ofmxm stochastic 
matrices. We consider the topology induced by the open sets in § m with respect to the Euclidean 
norm and the Borel sigma-algebra J£§m of this topology. We assume that we are given a probability 
space Pr(-)) and a measurable function W : O — > II^ S m . To every w 6 O, the function 

W(-) is assigning a discrete time process {W(k)}(u) in the countable product measurable space 
n^ (§ m , J^§m), where W(k) G § m is the random matrix of the process at time k. We refer to the 
process {W(k)}(u) interchangeably as a random chain or a random model and, when suitable, we 
suppress the explicit dependence on the variable u. We say that the chain {W-^/c)} is independent 
if the sigma algebras generated by the W(k)s for different k > are independent. If in addition 
W(k)s are identically distributed, then the model is independent identically distributed (i.i.d.). 

With a given random chain {W(k)}, we associate a linear discrete-time dynamic system of the 
following form: 

x(k + l) = W(k)x(k) for k = 0,1,2,..., (1) 

where x(k) G M m is a state vector at time k and x(0) is the initial state vector. We will often refer 
to the system in ([T]) as the dynamic system driven by the chain {W(k)}. 

We are interested in providing conditions guaranteeing that the dynamic system reaches a 
consensus almost surely. Since reaching the consensus is closely related to the ergodicity of the 
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chain, we are also interested in studying the ergodicity on the fundamental level. In our study 
of the random consensus and ergodicity, we use another property of the chain, an infinite flow 
property We start by providing these basic notions for a deterministic chain. 

Definition 1. Given a deterministic chain {G{k)} C § m , we say that: 

The system z(k + 1) = G(k)z(k) reaches a consensus if for any initial state z(0) G M. m , there exists 
a scalar c(z(0)) such that lim^oo \\z(k) — c(z(0)) e\\ = 0. 

The chain {G(k)} is ergodic if for any t > and j G [m], there is a scalar gj(t) such that 

lim §ij(k,t) = gj(t) for all i G [m], 

k— >oo 

where <$>(k,t) = G(k)G(k - 1) • • • G(t) for k > t and t > 0. 

The chain {G(k)} has infinite flow property if Y^kLo ^ies jes + Gji(k)) = oo for any 

nontrivial subset S C [m] . 

In the definition of the infinite flow property, the quantity J2ies jes (^j(^) + can be 

interpreted as a flow between the subset S and its complement S in a weighted graph. In particular, 
consider the undirected weighted graph Q{k) with the node set [m], the edge set induced by 
the positive entries in G(k) + G T (k), and the weight matrix G(k) + G T (k). Then, the quantity 
J2ies jes (Gij(k) + Gp(k)) represents the flow in graph Q{k) across the cut (S, S) for a nontrivial 
node set S C [in] and its complement S. For the graphs Q{k) induced by the matrices G(k), the 
infinite flow property requires that the total flow in time across any nontrivial cut (S, S) is infinite, 
which could be viewed as infinite information exchange between the nodes in S and S. 

The ergodicity is equivalent to the following condition [33]: for any t > and x G M. m , there is a 
scalar ^{t, x) such that limfc-^oo t)x = j(t, x) e. Since the matrices G(k) have finite dimension, 
the ergodicity is also equivalent to the following condition: for any t > and any £ G [m], there is 
a scalar ji(t) such that lim^oo $(A;, t)e£ = "fe(t) e. Also, due to the linearity and finite dimension 
of the system zik + 1) = G(k)z(k), the consensus can be studied by considering only the initial 
states x(0) = ee, £ G [m], rather than all x(0) G M m . 

Clearly, the ergodicity of the chain implies reaching a consensus. However, a consensus may be 
reached even if the chain {G(k)} is not ergodic, as seen in the following example. 

Example 1. Let G(0) = ev T for a stochastic vector v, and let G{k) = I for all k > 1. Then, 
we have $(/c,0) = ev T for all k > 1, implying that the system x(k + 1) = G(k)x(k) reaches a 
consensus. However, the chain {G(k)} is not ergodic since <&(k,t) = I for any k > t > 1. □ 

Using Definition [TJ we now introduce the corresponding events of consensus, ergodicity and 
infinite flow. Given a random chain {Ty(fc)}, let ^ denote the event that the system in (JTl) reaches 
a consensus for any initial state x(0). Let $ denote the event that the chain {^(A;)} is ergodic, 
and let & denote the event that the chain has the infinite flow property. We refer to $ and & 
as the consensus event, the ergodicity event and the infinite flow event, respectively. We say that 
the model is ergodic if the ergodicity event $ occurs almost surely. The model admits consensus if 
the consensus event ^ occurs almost surely. The model has infinite flow if the infinite flow event 
& occurs almost surely. The model has expected infinite flow if its expected chain {E[H / (A;)]} has 
infinite flow. 

3 Infinite Flow, Ergodicity and Consensus 

In this section, we further study the ergodicity event $, the consensus event ^ and the infinite flow 
event under different assumptions on the nature of the randomness in the model. In particular, 
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in Section 3.1 we establish some fundamental relations among $ , ^ and & . In Section 3.2 we 



investigate the 0-1 law properties of these events, while in Section [3T3] we provide some relations 
for a random model and its expected model. 

Throughout the rest of the paper, we use the following notation. For a stochastic matrix W 
and a nontrivial subset S C [m], we define W$ as follows: 

Ws= Yl (Wtf + Wii). (2) 

ieS,jES 

Note that for a given a random model {W(k)}, the infinite flow event is given by 

n |e^)=oo1. (3) 

Sc[m] lfc=0 J 

Note also that the infinite flow event requires that, across any nontrivial cut (S, S), the total flow 
in the random graphs induced by the matrices W(k) + W T (k), k > 0, is infinite. 

3.1 Basic Relations 

As discussed in Section |2j we have $ C ^ for any random model. We here show that the ergodicity 
event is also always contained in the infinite flow event, i.e., $ C & . We establish this by using 
the following result for a deterministic model. 

Lemma 1. Let {A(k)} C § m 6e a deterministic sequence, and let {z(k)} be generated by z(k + l) = 
A{k)z{k) for all k > with an initial state z(0) G W m . Then, for any nontrivial subset S C [m] 
and k > 0, we have 



maxzi(k + 1) < map s (°) + d(*(0)) ^i s (i)> 



t=o 

k 



minzAk + 1) > minz r (0) - d(^(0)) Y^Ajft), 

i6S r £ 5 ^ 

w/iere = max £eH y t - min reH y r for y G lR m . 

Proof. Let S C [m] be an arbitrary nontrivial set and let k > be arbitrary. Let z m in(&) = 
min rg [ m ] z r and <2 max (^) = max se[m] z s (k). Since + 1) = An{k)zi{k), by the stochasticity 
of A{k) we have Zi(fe) G [z m in(0), z max (0)] for all % G [m] and all fc. Then, we obtain for i G S, 

Zi(k + l) = J2 A u (k)z e (k) + Ai(k)z e (k) 
tes ees 



< 



( maxz s (A;) j + 2^(0)^4^) 
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where the inequality follows by Aa{k) > 0. By the stochasticity of A(k), we also obtain 

Zi(k + 1) < I 1 - A a( k ) ) maxz s (/c) 

= maxz s (k) + I z max (0) - max z s (k) ) A u (k). 

s&S \ s£5 / z — * 

By the definition of As in ([2f , we have < Ylees < As(&)- Since z maiX (0)—max seS z s (k) > 0, 

it follows 

Zi(k + 1) < max2: s (A;) + (z max (0) - max z s (/c))A s (/c) 
< max2 s (/r) + d(z(0))As(fc), 

where the last inequality holds since z max (0) — max se s z s (k) < z max (0) — Zmin(0) = d(z(0)). By 
taking the maximum over all i G S in the preceding relation and by recursively using the resulting 
inequality, we obtain max ie sZj(fc + 1) < max se s z s (k) + d(z(0))As(k) and recursively, we get 
max ie5 Zi(k + 1) < max seS z s (0) + d(z(0)) J2t=o A s(t)- 

The relation for mm.j e gz(k + 1) follows from the preceding relation by considering {z(k)} 
generated with the starting point —z(0). Q.E.D. 

Using Lemma [TJ we now show that the ergodicity event is contained in the infinite flow event. 

Theorem 1. Let {A(k)} C § m be an ergodic deterministic chain. Then YlT=o A s( k ) = 00 f or anv 
nontrivial S C [m] . In particular, we have $ C & for any random model. 

Proof. To arrive at a contradiction, assume that there is a nontrivial set S C [m] such that 
YlT=o A s{k) < oo. Since the matrices A{k) are stochastic, we have A$(k) > for all k. Therefore, 
there exists large enough t > such that Y^k=t A s(k) < \- 

Now, define the vector z = (z±, . . . , z m ) T , where Zi — for i G S and Zi — 1 for i G 5*. Consider 
the dynamic system z(k + 1) = A(k)z(k) for k > i, which is started at time t in state z(t) = z. 
Note that Lemma [T] applies to the case where the time t = t is taken as initial time, in which 
case d(0) corresponds to d(t) = maxj2j(t) — min,,- Zj(t). Also, note that d(t) = 1 by the definition 
of the starting state z. Thus, by applying Lemma [TJ we have for all k > t, max ig s Zi(k + 1) < 
m&x seS z s (t)+Y^ t= i A s(t) and mm je § Zj(k + 1) > min re § z r (t) ~YU=t 4s(0- Since max seS z s (t) = 
and min rg> § z r (t) = 1, it follows that max ie s Zi(k + 1) < Ylt=t A s(t) and min Jgi § Zj{k + 1) > 
1 — Y^t=t A s{t)- Using these relations and J2T=i A s(k) < |, we have liminffc_ s . 00 {zj{k) — Zi{k)) > 
1 — 2J2tLi A s(t) — \ f° r any j G S and i G 5*, thus showing that the chain {A{k)} is not ergodic 
- a contradiction. Therefore, we must have Yltlo A s{t) — 00 f° r any nontrivial S C [m\. 

From the preceding argument and the definitions of $ and J£~, we conclude that w 6 <f implies 
u; G & for any random model {W(k)}. Hence, $ C for any random model. Q.E.D. 

Theorem [T] shows that an ergodic model must have an infinite flow property. In other words, 
the infinite flow property of any random model is necessary for the ergodicity of the model. Later 
in Theorem [6j for a certain class of random models, we will show that the infinite flow is also 
sufficient for the ergodicity. 
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Figure 1: The relations among the ergodicity, consensus, and infinite flow events for a general random 
model. 

Figure [T] illustrates the inclusions $ C ^ and S C & for a general random model. The 
inclusion $ C ^ n & in Figure [TJ can be strict as seen in the following example. 



Example 2. Consider the 2x2 chain {A(k)} defined by 



A{0) 



1 i 

2 2 
1 1 



L 2 



and 



1 

1 



for k > 1. 



For any stochastic matrix B, we have BA(0) = A(0) and hence, the model admits consensus. 
Furthermore, the model has infinite flow property. However, the chain {A(k)}k>i does not admit 
consensus. Therefore, in this case c io n & = fi (the entire space of realizations) while $ — 0. □ 

Next, we provide a sufficient condition for the ergodicity and consensus events to coincide. 

Lemma 2. Let \W{ky\ be a (not necessarily independent) random chain such that each matrix 
W{k) is invertible almost surely. Then, we have S — % y almost surely. 

Proof. The inclusion $ C ^ follows from the definition, so it suffices to show ^ C $ almost surely. 
In turn, to show ^ C S almost surely, it suffices to prove that D R C $ for a set .R such that 
Pr(i?) = 1. For each k > 0, let be the set of instances u such that the matrix W(k)(u) is 
invertible. Define R = fl^ i?fc. We have Pr(Rk) = 1 for each k > by our assumption that each 
matrix W(A;) is invertible almost surely. In view of this and the fact that the collection {Rk} is 
countable, it follows that Pr(i?) = 1. 

We next show that ^ n R C S. Let w e ^ fl so that W(fe)(a;) has full rank for all 
k > 0. To simplify notation, let W{k) = W(k)(u). Consider an arbitrary starting time s > 0. 
We show that the consensus is reached for the dynamic z(t) = W(t — l)z(t — 1) with t > s, 
i.e., for any z(s) G M. m , we have lim^oo z(t) = ce for some c G R. For a given ^(s) G IR m , 
define x(0) = [W(s - 1) • ■ ■ lV(l)^(0)] _1 z(s) and consider the dynamic x(k) = W(k - l)x(k - 1) 
started at time to — with the initial vector x(0). By the definition of x(0), we have x(s) = 
W(s - 1) • • • W(l)W(p)x(0) = z(s). Therefore, for all t > s, 

x(t) = W(t-l)--- W(s - 1) ■ • ■ W(l)W(0)x(0) 
= W(t-1)---W(s)z(s) = z(t). 

By the definition of W(k), we have lim^oo x(t) = ce for some cGl (since u G ^). Therefore, it 
follows that lim^oo z(t) = ce, thus showing that the dynamic system z(t) = W(t— l)z(t— 1), t > s, 
reaches a consensus. Since this is true for arbitrary s > and z(s) G M m , the chain {W(k)}(u) is 
ergodic, which implies u G <f . Q.E.D. 

In general, there may be no further refinements of inclusion relations among the events &, & 
and ^ even when the model is independent, as indicated by the following example. 
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Example 3. Consider an independent random model where for p G (0, 1], we have W(0) = ^ee T 
with probability p, W(0) = I with probability 1 — ps and W(k) = I with probability 1 for all k > 1. 
In this case, the consensus event & happens with probability p > 0. However, the infinite flow and 
the ergodicity events are empty sets. □ 

Example [3] shows that we can have D <§ = and fl & = 0, while ^ 0. Thus, even for an 
independent model the consensus event need not be contained in either $ or & . However, if we 
further restrict our attention to i.i.d. models, we can show that $ = c € almost surely. To establish 
this, we make use of the following lemma. 

Lemma 3. Let A G S m and x G IR m . Also, let A be such that max ig [ m ] [Ae^i — mhij g [ m ] [Aei]j < 
for any I G [m], where [v]i denotes the ith component of a vector v. Then, we have maXj[Ar]j — 
minjfAcJj < | for any x G [0, l] m . 

Proof. Let x G IR m with Xi G [0, 1] for any I G [m]. Then, we have for any i,j G [m], 

m m 

y* ~ Hi = y^S A it ~ A jt)^t < \ Aii ~ A i £ \ 
l=i i=\ 

m 

= ^2\[Aee] i -[Ae i ) j \. 
i=i 

By the assumption on A, we obtain |[Ae^]j — [Ae^]j| < maXi 6 [ m ][Ae^]j — minj 6 [ m ][Ae^]j < Hence, 
Vi ~ Vj ^ I2?=i 2^ = h impiying max^ - min,, yj < \. Q.E.D. 

We now provide our main result for i.i.d. models, which states that the ergodicity and the 
consensus events are almost surely equal. We establish this result by using Lemma [3] and the 
Borel-Cantelli lemma (see [34], page 50). 

Theorem 2. We have <S = c € almost surely for any i.i.d. random model. 

Proof. Since S C the assertion is true when consensus occurs with probability 0. Therefore, it 
suffices to show that if the consensus occurs with a probability p other than 0, the two events are 
almost surely equal. Let Pr(^) = p with p G (0, 1]. Then, for all u G ^ , 

lim d(x(k))(u) = where d(x) = maxij — minxj, 

fc— >oo i j 

and {x(k)}(co) is the sequence generated by the dynamic system Q with any x(0) G M. m . 

For every £ G [m], let {x e (k)} be the sequence generated by the dynamic system in ([I]) with 
x(0) = e£. Then, for any u G there is the smallest integer K {u) > such that 

d(xHk))(uS) < — for all k > KHuS). 
2m 

Note that d(x (k)){u) is a nonincreasing sequence (of k) for each £ G [m]. Hence, by letting 
K{oj) = max£ g [ m ] K e (u) we obtain d{x e {k)){u>) < for all t G [m] and k > K(oS). Thus, by 
applying Lemma [3j we have for almost all u G 

d(x(k))(u) < i (4) 
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for all k > K(oS) and x(0) G [0, l] m . By the definition of consensus, we have liniiv-s>oo Pr(K < 
N) > Pr(^) = p. Thus, by the continuity of the measure, there exists an integer N\ such that 
Pr(K < JVi) > |. ' 

Now, let time T > be arbitrary, and let denote the iVi-tuple of the matrices W(s) driving 
the system g} for s = T + Nik, . . . , T + Ni(k + 1) - 1 and k > 0, i.e., 

l T k = (W{T + iVi/c), W(T + Nik + 1), 

...,W(T + Ni(k+1)-1)) for all fc > 0. 

Let Ltv denote the collection of all iV-tuples (Ai,...,An) of matrices A t G S m , i G [iV] such 
that for x(N) = A N A N _i ■ ■ ■ Aix{0) with x(0) G [0, l] m , we have d(x(N)) < §. By the definitions 
of and Ltv, relation @ and relation Pr(i^ < N x ) > \ state that Pr({/^ G L^}) > 2. By 
the i.i.d. property of the model, the events {1% G L^}, k > 0, are i.i.d. and the probability of 
their occurrence is equal to Pr({/^ G L^}), implying that Pr({/^ G L^}) > f for all A; > 0. 
Consequently, YlkLo ^ r ({^k e ^nA) = oo. Since the events {1% G ^ati} are i.i.d., by Borel-Cantelli 
lemma Pr({w 6 fi | w G {/f G IjvJ i.o.}) = 1, where i.o. stands for infinitely often. Observing 
that the event {w G ft | u G {lj G L Nl } i.o.} is contained in the consensus event for the chain 
{W(T + k)}k>o, we see that the consensus event for the chain {W(T + k)}k>o occurs almost surely. 
Since this is true for arbitrary T > it follows that the chain {^(A;)} is ergodic a.s., implying 
C g a.s. This and the inclusion S C yield ^ ' = S a.s. Q.E.D. 

Theorem |2] extends the equivalence result between the consensus and ergodicity for i.i.d. models 
given in Theorem 3. a and Theorem 3.b of [5] (and hence Corollary 4 in [5]), which are established 
there assuming that the matrices have positive diagonal entries almost surely. The relations among 
ff, $ , and & for i.i.d. case are illustrated in Figure [2] 

3.2 0-1 Laws 

In this section, we discuss 0-1 laws for the events 6 & and c io for independent random models. The 
0-1 laws specify the trivial (or 0-1) events, which are the events occurring with either probability 
or 1. The ergodicity event is a 0-1 event, as shownQin [3], Lemma 1. Since the ergodicity event is 
always contained in the consensus event, the ergodicity event S occurs with probability whenever 
the consensus event c io occurs with a probability p G (0, 1). In other words, we may have Pr(#) = 1 
only if Pr(^f) = I. 

We next show that the infinite flow is also a 0-1 event. 

Lemma 4. For an independent random model, the infinite flow event & is a 0-1 event. 

Proof. For a nontrivial S C [m], the sequence {Ws(k)} of undirected flows across the cut (S, S) (see 
Eq. Q) is a sequence of independent (finitely valued) random variables. The event {YlkLo Ws(k) = oo} 
is a tale event and, by Kolmogorov's 0-1 law (|34j. page 61), this event is a 0-1 event. Since there 
are finitely many nontrivial sets S C [m], the event & = DscFm] {^fclo Ws(k) = oo} is also a 0-1 
event. Q.E.D. 

^^Even though the result there was stated assuming a more restrictive random model, the proof itself relies only 
on the independence property of the model. 
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Infinite Flow 
0-1 event 



Figure 2: The ergodicity and consensus coincide a.s., and all three events assume 0-1 law for an i.i.d 
model. 

While both events $ and & are trivial for an independent model, the situation is not the same 
for the consensus event . In particular, by Example [3] where p E (0, 1), we see that the consensus 
event need not assume 0-1 law since it can occur with a probability p e (0, 1). 

However, the situation is very different for i.i.d. models. In particular, in this case the consensus 
event is also a trivial event, as seen in the following lemma. 

Lemma 5. For an i.i.d. random model, the consensus event ^ is a 0-1 event. 

Proof. The result follows from the fact that S is a trivial event and Theorem |2j which states that 
$ = almost surely for i.i.d. models. Q.E.D. 

Figure [2] illustrates the 0-1 laws of $, e & and & for an i.i.d. model. Our next example demon- 
strates that the inclusion relation in Figure [2] can be strict. 

Example 4. Consider the independent identical random model where each W(k) is equally likely 
to be any of the m x m permutation matrices. Then, in view of the uniform distribution, we have 
E[W(k)] = i ee T for all k. Hence, by Theorem 5j it follows that the infinite flow event & is 
happening almost surely. But, since the chain \W{k)} is a sequence of permutation matrices, the 
consensus event ^€ never happens. □ 



3.3 Random Model and Its Expected Model 

Here, we investigate the properties of an independent random model and its corresponding ex- 
pected model. We establish two results in forthcoming Theorems [3] and [4] that later on play an 
important role in the establishment the Infinite Flow Theorem in Section [5] The first result shows 
the equivalence of the infinite flow property for a random chain |W(A;)} and its expected chain 
{E[W(A;)]}, as given in the following theorem. 

Theorem 3. Let \W(ky\ be an independent random model. Then, the model has infinite flow 
property if and only if the expected model has infinite flow property. 

Proof. Let S C [m] be nontrivial. Since the model is independent, the random variables Ws(k) 
are independent. By the definition of Ws(k) and the stochasticity of W(k), we have < Ws(k) < 
Y^hLi Y^j=\ W%j(k) = m for all k > 0. Thus, by monotone convergence theorem ([SI], page 225), 
the infinite flow property of the model implies the infinite flow property of the expected model. On 
the other hand, since the model is independent and < Ws(k) < m, by Kolmogorov's three-series 
theorem ([35, page 64) it follows that: if E[W s (k)} = oo, then PrQXo^O) = °°) > °- 

Since Yl°k=oWs{k) is a trivial event, we have Pr (X^o Ws{k) = oo) = 1. Thus, since S C [m] is 
arbitrary, the model has infinite flow property. Q.E.D. 
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There is no analog of Theorem [3] for the ergodicity or consensus event, unless additional as- 
sumptions are imposed. However, a weaker result holds as seen in the following. 

Lemma 6. Let {W(k)} be an independent model and assume that the model admits consensus (is 
ergodic). Then, the expected model {E[W(k)}} reaches a consensus (is ergodic). 

The prooQ of Lemma [6] can be found in [5J E] • 

The following theorem states another important result for later use. As a consequence of 
Lemma |6j and Theorems [I] and |3j the result provides an equivalent deterministic characterization 
of the ergodicity for a class of independent models. 

Theorem 4. Let {W(k)} be an independent random model such that $ — almost surely. Then, 
the model is ergodic if and only if the expected model is ergodic. 

Proof. If the ergodicity event is almost sure, then by Lemma |6j the expected model is ergodic. 
For the converse statement, let the chain {E[W / (/c)]} be ergodic. Then, by Theorem [l] the chain 
{E[W(fc)]} has infinite flow. Therefore, by Theorem[3]the infinite flow event is almost sure, and 
since $ — ^ a.s., the ergodicity event is almost sure. Q.E.D. 

4 Model with Feedback Property and Steady State in Ex- 
pectation 

In this section, we discuss two properties of a random model that are important in the development 
of our main results in Section [5] In particular, we introduce and study a model with feedback 
properties and a model with a common steady state in expectation. Stronger forms of these 
properties have always been used when establishing consensus both for deterministic and random 
models. Here, we provide some new fundamental insights into these properties. 

4.1 Feedback Properties 

We define several types of feedback property. Recall that W % denotes the ith. column vector of a 
matrix W . 

Definition 2. A random model {W{k)} has strong feedback property if there exists 7 > such 
that 

Wa(k) > 7 a.s. for all k > and all i G [m]. 
The model has feedback property if there exists 7 > such that 

E[W H (k)W ij (k)]> 1 E[W ll (k)], 

for all k > 0, and all i,j G [m] with i 7^ j . The model has weak feedback property if there exists 
7 > such that 

E[{W\k)) T W j {k)\ > 1 {E[W ij {k)} + tfWiiik)}) 
for all k > 0, and all i,j G [m] with i ^ j. The scalar 7 is referred to as a feedback constant. 



2 Assuming more restrictive assumptions on the model, the result of the lemma was stated in [5] (Theorem 3, 
(a) =>■ (6)) and jF] (Remark 3.3). However, the proofs there rely only on the independence of the model. 
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While the difference between feedback and strong feedback property is apparent, the difference 
between weak feedback and feedback property may not be so obvious. The following example 
illustrates the difference between these concepts. 

Example 5. Consider the static deterministic chain {A(k)}: 



A(k) = A 



i i 

u 2 2 

1 * 

2 u 2 

* 1 

L 2 2 



for k > 0. 



Since A^A^ = and A^ ^ for all i ^ j , the model does not have feedback property. At the same 
time, since Ay + Aji = 1 for i ^ j , it follows (A l ) T A> = \ = |(A'j + A?'*)- Thus, {A(k)} has weak 



feedback property with 7 = 7.0 

It can be seen that strong feedback property implies feedback property, which in turn implies 
weak feedback property. The deterministic consensus and averaging models in [191 IZH [261 129] 
require that the matrices have non-zero diagonal entries and uniformly bounded non-zero entries, 
which is more restrictive than the strong feedback property. 

We next show that the feedback property of a random model implies the strong feedback 
property of its expected model. 

Lemma 7. Let a random model {W(k)} have feedback property with constant^. Then, its expected 
model {E\W(k)]} has strong feedback property with 

Proof. Let the model have feedback property with a constant 7. Then, by the definition of the 
feedback property, we have E[Wa(k)Wij(k)] > r yE[Wij(k)) for any k > and i, j G [m] with i ^ j. 
Since Wij(k) < 1, it follows that 

E[W u (k)\ > EiWaWWyik)] > 1 E[W ij {k)\ 

for all i,j G [m],i 7^ j. The matrices W(k) are stochastic, so that we have Y^jLi = 1- 

Hence, for every k > and i G [m], there exists an index j* (the dependence on k and i is 
suppressed) such that E[W / jj«(/c)] > — . If j* = i, then we are done; otherwise we have E[Wjj(fc)] > 
7 E[Wjj*(A;)] > — for all i G [m]. Hence, the expected chain has the strong feedback property with 
constant — . Q.E.D. 

We now focus on an independent model. We have the following result. 

Lemma 8. Consider an independent model {W(k)}. Suppose that the model is such that there is 
an rj > with the following property: for all k > and i,j G [in] with i 7^ j , 



E[W^k)] > 

or the following property: 

for all k > and i,j G [m] with i 7^ j , 



E[W«(A0Wii(A0] > V, 



E[W t3 {k)} > 



(w^krfw^k) 



> rj. 



Then, respectively, the model has feedback property with constant rj or weak feedback property with 
constant r)/2. 
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Proof. We prove only the case of feedback property, since the other case uses the same line of 
argument. If EjW^/c)] = for some k and i,j with i ^ j, then the relation E[Wu(k)Wij(k)] > 
rj E[Wij(k)] is satisfied trivially (with both sides equal to zero). If E[Wjj(/c)] > 0, then by the 
assumption of the lemma, we have E[Wij(k)Wu(k)} > rj. Furthermore, since 1 > Wij(k) for all 
i,j and k, it follows that E[Wij(k)Wn(k)} > rj > r)E\Wij(k)] , thus showing that the model has 
feedback property with constant rj. Q.E.D. 

The i.i.d. models {H^A;)} with almost surely positive diagonal entries Wa(k) have been studied 
in [HI El E]- Such models have feedback property as seen in the following corollary. 

Corollary 1. If {W(k)} is an i.i.d. model with almost surely positive diagonal entries, then the 
model has feedback property with constant 

7 = min {i ^ mw .. ik)]>0} E[U / ii (/c)H / ij (A;)]. 

Proof. Let E[Wjj(A;)] > for some i,j G [m). Since Wu{k) > a.s. and Wij(k) > 0, we have 
^■[Wii(k)Wij(k)} > 0. Define r/ = mm{j^|^..(&)] >0 } E[Wii(k)Wij(k)]. Since the model is i.i.d., the 
constant rj is independent of time. Hence, by Lemma [8] it follows that the model has feedback 
property with constant 77. Q.E.D. 

4.2 Steady State in Expectation 

Here, we consider a model with another special property. Specifically, we discuss a random model 
{H^/c)} such that its expected chain {E[W(fc)]} has a common steady state. 

Definition 3. A random model has a common steady state in expectation if there is a stochastic 
vector 7r€l m such that TT T E[W(k)} = ty t for all k. 

For example, the mxm matrices that are doubly stochastic in expectation satisfy the preceding 
definition with ir = — e, such as the matrices arising in a randomized broadcast or gossip over a 
connected (static) network [TBI ITT] . 

Consider the function given by 

m 

v(x) = J2^ ( Xi - 7rTx ) 2 for x G Rm - ( 5 ) 

i=l 

The function V(x) measures the weighted spread of the vector x entries with respect to the weighted 
average value n T x. 

We at first study the behavior of the weighted averages ir T x(k) along the sequence {x(k)}. The 
main observation is that the random scalar sequence {ir T x(k)} is a bounded martingale, which 
leads us to the following result. 

Lemma 9. Let \W(k)} be an independent random model with a common steady state it in expec- 
tation. Then, the sequence {n T x{k)} converges almost surely for any x(0) G M. m . 

Proof. By the model independency and 7r r E[H^(&;)] = 7r T , it follows that the process {7r T x(k)} 
is a martingale with respect to the natural filtration of the process for any initial x(0) G IR m . 
Since the matrices W{k) are stochastic, the sequence {x{k)} is bounded. Thus, {n T x(k)} is 
a bounded martingale. By the martingale convergence theorem (see [35], Theorem 35.5), the 
sequence {7i T x(k)} converges a.s. Q.E.D. 
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We next characterize the limit of the martingale {ir T x(k)}. Let Aj j7r G K be the limit of the 
martingale {7T T x(k)} for the initial state x(0) = e$, and let A w be the vector defined by 

Ayr — (Ai )7r , • • • , A mj7r ) (6) 
with A i)7r = lim n T W(k) ■ ■ ■ W(0)ei for i e [m]. 

k—>oo 

In the following lemma, we provide some properties of the random vector A„-. 

Lemma 10. Let {W(k)} be an independent model with a common steady state n in expectation. 
Then, the random vector X n has the following properties: 

(a) linifc^oo ir T x(k) = A^x(O) a.s. for any x(0) G M. m . 

(b) X n is a stochastic vector. 

(c) E[K}=n. 

(d) For every x(0) G W 71 , the limit points of the sequence {x(k)} lie in the random hyperplane 
'H~?r,x T x(o) = {x E M. m | ti t x = X^x(0)} almost surely. 

Proof. By Lemma [9] and the definition of A^, we have lim^oo 7i T x(k) = X in almost surely for 
initial state e,. Using the linearity of the system in 0, we obtain for any x(0) G M. m , 



in. 

lim 7r T x{k) = V Xi(0) ( lim 7r T W(k - 1) ■ ■ ■ W(0)e i ) 

8=1 

m 

= J2^(0)X hn = A^x(O), 
i=i 

thus showing part (a). 

Since x{k) = W(k - 1) • • • W(0)x{0), by part (a) it follows lim*-** ic T W(k -!)■■■ W(0) = X?. 
The matrices W(k) and the vector ir have nonnegative entries implying that the vector A^ also has 
nonnegative entries. By letting x(0) = e and using the stochasticity of W(k), we have x(k) = e for 
all k, implying 1 = ir T x(k) for all k. Thus, we have 1 = lim^oo ir T x(k) = X^x(0) = A^e, where 
the second equality holds by part (a). Hence, A^ is a stochastic vector. 

To show part (c), we note that by the martingale property of the process {7r T x(k)}, we have 



E[n T x(k) 
lim^oo E 



7l T X[ 



7r T x(0) for all k > and x(0) G R m . By the boundedness of the martingale, we have 
'(k)\ = E[A^] x(0) for any x(0) G IR m . The preceding two relations imply E[A„-] = n. 
For part (d), we note that the sequence {x(k)} is bounded for every x(0) G M. m by the stochas- 
ticity of W(k); thus, it has accumulation points. By part (a), each accumulation point x* of the 
sequence satisfies ti t x* = A^x(O) a.s. Q.E.D. 



We now focus on the sequence {V(x(k))}. We show that it is a convergent supermartingale, 
which indicates that V(x) is a stochastic Lyapunov function for the random system in (jl|). 

Theorem 5. Let the random model {W(k)} be independent with a common steady state it in 
expectation. Then, we almost surely have for all k > 0, 

E[V(x(k + 1)) I x(k)\ (7) 

< v{x(k)) - H a( k ) - x A k )) 2 > 

i<j 

where H{k) = E\W T (k)DW{k)\. Furthermore, {V(x(k))} converges almost surely. 
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Proof. By using D = diag(ir), from the definition of the function V(x) in ^ we have 

V(x) = x T (I - ire T )D(I - en T )x = x T (D - nn T )x, 

where the second equality is obtained by using e T D = n T , De = n, and n T e = 1. In view of 
x(k + 1) = W(k)x(k), it follows that for all k > 0, 



V(x(k + 1)) = X(k + lf(D - TTTT T )x(k + 1) 



x{k)W{k) T {D 



7T7T 



)W{k)x{k). 



(8) 



Since the model is independent, by taking the expectation conditioned on x(k), we obtain 

E[V{x{k + 1)) | x(k)} = x{k) T E[W{k) T {D - mi T )W(k)] x(k), 
almost surely for all k > 0. Since H(k) = E[W T (k)DW(k)], we further have 

E[V(x(k + l)) | x(k)] 

= x{k) T H{k)x{k) - E\(iT T W{k)x{k)) 2 | x(k) 
< x{k) T H{k)x{k) - (E[n T W(k)x(k) | x(k)]) 2 , 

where the inequality follows by Jensen's inequality (see [31], page 225) and the convexity of the 
function s i— > s 2 . The expected matrices E[W / (/c)] have the same steady state it, implying that 
almost surely for all k > 0, 

E[7T T W{k)x(k) | x(k)] = 7i T E[W{k)}x{k) = 7c T x{k). 

By combining the preceding two relations, we see that almost surely for all k > 0, 

E[V(x(k + 1)) | x(k)} < x{k f (H{k) - 7rvr T ) x(k). 

By adding and subtracting x{k) T Dx{k) to the right hand side of the preceding relation and us- 
ing ([8|, we obtain almost surely 



E[V{x{k + l)) | x{k)\ 

< x(kf (H(k) - D) x(k) + V(x{k)) for all k > 0. 



(9) 



Now, we show that x{k) T (H(k) — D) x{k) = J2i<j Hij(k)(xi(k) — Xj{k)) 2 . By the definition of 
H{k) we have Hij(k) = YleLi ^■[ K (W£i(k)Wjij(k)], so that for i e [m], 



Y,H VJ {k) = E 



i=i t=\ 



3 = 1 
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Since W(k) is stochastic, we have YlT=ij^i ^tj{ k ) = 1 — Wa(k), implying that 



3=1 



1=1 



e=i 



iii - Hu(k), 



where the last equality follows from E[7r T W / (/c)] = 7r T . Since D = diag(n), the preceding relation 
yields 



Hu(k) = Da - ^THij(k). 



(10) 



3=1 



Therefore, for any x G M m , we have x H(k)x = x « Sjli Hij( k ) x ji which can be further 

written as 



m m 



x T H(k)x = y^jfjHj^fyxj + y j x i H ii (k)x i 

i=l i=i i=l 
mm m 
^ ^ X% ^ ^ Hij{k^{x 'j Xj) -|- ^ ^ XiDnXi ) 



i=l i=l 



1 = 1 



where the last equality follows from relation (10). The matrix H(k) = E[W (k) T DW (k)] is sym- 
metric, so that 



m m 



J2 x i^2 H ij( k )( x j =-^2H ij (k)(x i -x j ) 2 , 

i=l 3=1 i<j 

3=£i 

implying that x T H(k)x = — Yli<j Hij( k )( x i ~ x j) 2 + x T Dx. Therefore, we have 

:{k) T {H{k) - D) x(k) = -J2 H n( k ) ( x i( k ) - x A k )f ■ 



x{ 



i<j 



By combining the preceding relation with ([9]), we conclude that relation (J7|) holds a.s. for all k > 0. 

Since each W(k) is a stochastic matrix and 7r is stochastic vector, the matrix H(k) has non- 
negative entries for all k. Hence, from the preceding relation it follows that {V(x(k))} is a su- 
permartingale. The convergence of {V(x(k))} follows straightforwardly from the nonnegative 
super martingale convergence (see [34J, (2.11) Corollary, page 236.) Q.E.D. 

We conclude this section with another result for the weighted distance function V(x). This 



result plays crucial role in establishing our result in Section 5.2 



Lemma 11. Let ir G K m be a stochastic vector, and let x G M. m be such that X\ < • • • < x m . Then, 
we have 



[m 



1 1 

— — V(x) < y2( x i+i - Xi) 

- I 2 Xm-Xl ~ 

' i=l 
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Proof. We establish two relations 



m 



j m— 1 



TO— 1 

E 

i=i 



m — 1 



< 



=i 

TO — 1 

E 

i=l 



(11) 

(12) 



Observe that the desired result follows from (11 )— (12). 

We now show relation (11). We have Xi < x m for all i. Since 7r is stochastic, we also have x m > 
n T x > Xi. Thus, V(x) = Y!iLi' K i( x i~' KTx ) 2 - (^m-^i) 2 - By writing x m - Si = ^X^O^+i 
we obtain 

/ 1 m-l \ 2 

(x m - Xi) 2 = (m - l) 2 y^(x i+1 - x^ 

m— 1 

< (m - 1) 2j(x i+ i - x^ 2 , 
i=i 

where the last inequality holds by the convexity of the function s H- s 2 . Using V(x) < (x m — Xi) 2 
and the preceding relation we obtain relation (11). 

To prove relation fl2] ), we write (x m -Xi) Y^™~ (x i+1 -Xi) 2 as E^E/^+i-x,-) ^"^(xj+i-Xj) 2 
, which is equal to J27=i( x j+i ~~ x j) 3 + ^ with A given by 



To estimate A, we consider scalars a > and (3 > 0, and let w = (a, (3) and t> = (/3 2 ,a 2 ). Then, 
by Hole 
Hence, 



by Holder's inequality with p — 3, q — |, we have u T t> < ||tt|| p ||f where || • \\ p is the p-norm. 



a/3 2 + (3a 2 < (a 3 + (3 3 ) 5 ((3 3 + a 3 ) 5 = a 3 + (3 3 . 



(13) 



By using (13) with aj = (xj + i — Xj) and (3i = (x i+ i — Xi) for different indices j and i, 1 < j < i < 
m — 1, we obtain 



m— 1 



8=1 



m— 1 



j=l j<i 

TO — 1 

= (m-l) y^Xj+t - Xi) 3 , 

i=l 



which completes the proof. Q.E.D. 
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5 Model with Infinite Flow Property 



We consider an independent random model with infinite flow. We show that this property, together 
with weak feedback and a common steady state in expectation, is necessary and sufficient for 
almost sure ergodicity. Moreover, we establish that the ergodicity of the model is equivalent to 
the ergodicity of the expected model. 



5.1 Preliminary Result 



We now provide an important relation that we use later on in Section 5.2 



Lemma 12. Let {A(k)} C S m and z(k + 1) = A(k)z(k) for all k > and some z(0) G W m . Let 
a be a permutation of the index set [m] corresponding to the nondecreasing ordering of the entries 
ze(0), i.e., a is a permutation on [m] such that z ai (0) < • • • < z am (0). Also, let T > 1 be such that 



T-l 



^^As(k) > 5 for every S C [m], (14) 



k=0 



where 5 G (0, 1) is arbitrary. Then, we have 



T-l 

k=0 i<j 

6(1 -5) 2 "'~ L 
^ m (0)-z ffl (0) 



> — — — - ^(0)) 3 . 



Proof. Relation (14) holds for any nontrivial set S C [m\. Hence, without loss of generality we 
may assume that the permutation a is identity (otherwise we will relabel the indices of the entries 
in z(0) and update the matrices accordingly). Thus, we have ^i(O) < ••• < z m (0). For each 
£= 1, ... ,m — 1, let Si = {1, ... ,£} and define time ti > 1, as follows: 

t e = argmm <^ > A Se (k) > 5 — — 

Since the entries of z(0) are nondecreasing, we have 8 ^ +1 (o)-zi(o)) — 8 for all £ = 1, . . . , m — 1. 
Thus, by relation (14), the time tt > 1 exists and ti<T for each £. 

We next estimate Zj(k) — Zi(k) for all i < j and any time k = 0, • • • ,T — 1. For this, we 
introduce for < k < T — 1 and i < j the index sets aij(k) C [m], as follows: 

aij (k) = {£ G [m] | k < U - 1, £>i, £+!<]}. 

Let k < ti — 1 for some £. Since = {1, . . . , £}, we have % G and j G Thus, by Lemma [l] 
we have for any k > 1, 

fe-i 



«i(A;) < maxz s (0) + (z m (0) - ^(0)) V A s< (r) 



T=0 
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k-1 



z 3 {k) > minz r (0) - (> w (0) -*i(0)) V^(r). 
re5f r=0 

Furthermore, max sg s, -Zs(O) = ^(0) and mm r& § e z r (0) = 27+1(0) since Se = {1, . . . ,£} and zi(0) < 
• • • < ^m(O). Thus, it follows 

fc-i 

«i(A:)-^(0) < (z m (0) - zi(0))^2 A St (r), 

r=0 
fc-1 



z /+1 (0) - < (^(0) - Zl (0)) 5^A S( 



T=0 



By the definition of time tg, we have (z m (0) — zi(0)) ^2 T=Q As e (r) < 5 (zf+i(0) — Z|(0)) for A; < t^ — 1. 
Hence, by using this and the definition of a,ij(k), for any £ G a,ij(k) we have 

Zi(k) - z t (0) < S(z t+1 (0) - z t (0)), (15) 
z m (0) - Zj(k) < S(z e+1 (0) - Zi (0)). (16) 

Now suppose that atj(k) = {£i, . . . ,£ r } for some r < m — 1 and £i < • ■ • < £ r . By choosing 



i in (15) and £ = £ r in (16), and by letting an = z i+ i(0) — Zi(Q), we obtain 



Zj(k) - Zi(k) > 2:4+1(0) - z tl (0) - 5(ae r + a h ). 

Since ^(0) < z i+1 (0) for alH = 1, . . . , m - 1, we have 2^(0) < ^ 1+ i(0) < • • • < 2^(0) < 24+1(0), 
which combined with the preceding relation yields Zj(k) — Zi{k) > Y^=i( z £(.+i(^) — 2^(0)) —5(ag r + 
a^). Using = 2j+i(0) — 2^(0) and = {£1, . . . ,£ r }, we further have 



(k) - Zj(k) >y]ae e - 8{a tr + at^) (17) 



r-l 



By Eq. [T7j it follows that 



i<3 

> {i - 5fY, (M k ) + M k )) [ E a < 



>(i-*) 2 E^(*)+^i(*)) E 



where the last inequality holds by an > 0. In the last term in the preceding relation, the coefficient 
of aj is equal to (1 — S) 2 Ag e (k). Furthermore, by the definition of ciij(k), we have £ G a^ik) only 
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when k < t e — 1. Therefore, 

i<j 

= (l-5) 2 E A 5£ (Jfc)a|. 

{£|fc<t,-l} 

Summing these relations over k — 0, . . . , T — 1, we obtain 

T-l 

>(l-*) 2 £ E A ^(kW e 
k=o {e\k<t e -i} 

m-l /te-1 \ 

>a- 5 ) 2 E E^w K 

£=1 \fe=0 / 

where the last inequality follows by exchanging the order of summation. By the definition of te 
and using a e = z t+1 (0) - z t (0), we have Ytk^l A s e (k) > Zm(0 fi l(0 ) , implying 

T-l 



k=0 i<j 



m—1 



Q.E.D. 



5.2 Sufficient Conditions for Ergodicity 

We now establish one of our main results for independent random models with infinite flow property 
and having a common vector in expectation and weak feedback property. 
Let to = and for any q > 1, let 



t-i 



t q = argmin Pr min Ws(k) > 5 \ > e, 

t>t q . 1+ l \Sc[m] ^ 



(18) 



where e, 5 G (0,1) are arbitrary. Define 



CO 



tq+l- 1 

min E Ws(k){u) > 5 

SG[m] f-f 

K — In 



> for q > 0. 



(19) 
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Since the model has infinite flow property, the infinite flow event & occurs a.s. Therefore, the 
time t q is finite for all q. 

We next show that either the infinite flow or expected infinite flow property is sufficient for the 
ergodicity of the model. 

Theorem 6. (Sufficient Ergodicity Condition) Let {W(k)} be an independent random model with 
a common steady state 7T > in expectation and weak feedback property. Also, let the model have 
either infinite flow or expected infinite flow property. Then, the model is ergodic. In particular, 
limfc^oo Xi(k) = A^x(O) almost surely for all i G [m], where X w is the random vector of Eq. 

Proof. Assume that the model has infinite flow property. Let x(0) G M. m be arbitrary initial state. 
Let us denote the (random) ordering of the entries of the vector x(t q ) by rj q for all q. Thus, at 
time t q , we have x v v(t q ) < ■ ■ ■ < x^itq). 

Now, let q > be arbitrary and fixed, and consider the set s# q in (19). By the definition of s/ q , 
we have Y^k=t 1 Ws(k)(u) > 5 for any S C [m] and u G &/ q . Thus, by Lemma 12, we obtain for 
any uj G s/ q , 



tq + l-l 

E E m {t) + w 0l {t))Mt)-^m 2 ^) 

t=t q i<j 



where d(t q ) = x^^tg) — x v ^(t q ) and the last inequality follows by Lemma 11 We can compactly 
write the inequality as: 



E E + w nV)) o*(*) - x At)) 2 

t=t q i<j 

> V{x{t q ))l^ (20) 

where is the indicator function of the event $$ q . 

Observe that xit) and Wit) are independent since the model is independent. Therefore, by 
Theorem [5j we have 

E[V(x(t q+1 )) ~ V(x(t q ))} 

tq-l 

<-EE^(*)e[(^)-^)) 2 ], 

t=t q i<j 

with H{k) = E[W T (k) DW (k)~\ and D = diagijz). Let 7r min = min^ g [ m ] ii£ and note that 7r min > 
by 7r > 0. Thus, 

H ll {t) = E[{W\t)) T DW\t)] 

>-K^E[{W\t)) T W\t)\ 
>7r min7 (EiWijit)} + E[W,i(f)]) . 
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Therefore, 



E[V(x(t q+1 ))} - E[V(x(t q ))} 



< -7T rl 



,7E 



t a -l 



E E (*) + (**(*) - ^c*)) 1 



Further, by using relation (20), we obtain 



E[V(x(t q+1 ))} - E[V(s(f s ))] 
"<5(l-<5) 2 



< -vr min 7 E 



lA q V{x{t q )) 



< 



eS(l - S) 2 ^ 
(m-1) 2 



E[V(x(t q ))] 



where the last inequality follows by Pr(&/ q ) > e, and the fact that and V(x(t q )) are independent 
(since x(t q ) depends on information prior to time t q and the set srf q relies on information at time 
t q and later). Hence, it follows 



crs - i, mi / (i e5(l - 5) 2 77r min 
E[V{x{t q+ i))\ < 1 - 



Therefore, for arbitrary g > we have 



E[v(*(g)] < i 



(m — l) 2 



e5(l - 5) V, 



E[F(*(g)] . 



E[V(x(0))] 



implying that ^^L E[V(a;(t 9 ))] < oo. In view of the nonnegativity of V(x), by the monotone con- 
vergence theorem ([US], Theorem 16.6) it follows E Ylq^Lo V( x (tq)) < 00 > implying lim^oo V(a;(i ? )) 
a.s. According to Theorem |5j the sequence {V(x(k))} is convergent, which together with the 
preceding relation implies that lim^oo V(x(k)) = a.s. 

To show the ergodicity of the model, we note that by the convergence result for the martingale 
{ir T x(k)} in Lemma 10|a), we have lim^oo n T x(k) = A^x(0) a.s., where the random vector X n is 
given by Eq. @. Now, using ir T x{k) — > A^x(0) and the fact that all norms in R m are equivalent, 
we obtain lim^oo \xi{k) — A^r(0)| = a.s. for all i G [m], thus showing the ergodicity of the 
model. 

Assume now that {VT(A;)} has expected infinite flow, i.e., J2T=o E[Ws(k)} = oo for any nontrivial 
S C [m]. By Theorem |3j {M^(/c)} has infinite flow property if and only if it has expected infinite 
flow, and the result follows by the preceding case. Q.E.D. 



5.3 Necessary and Sufficient Conditions for Ergodicity 

Here, we provide the central result of this paper. The result establishes necessary and sufficient 
conditions for ergodicity of random models with weak feedback property and a common steady 
state it > in expectation. The conditions are reliant on infinite flow, and guarantee that the 
ergodicity of the model is equivalent to the ergodicity of the expected model. The result emerges as 
an outcome of several important results that we have developed so far. In particular, we combine 
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the result $ C & stating that the ergodicity event is always contained in the infinite flow event 
(Theorem [I]), the deterministic characterization of the infinite flow of Theorem [3j and the sufficient 
conditions of Theorem [6j We also make use of Theorem [4] providing conditions for equivalence of 
the ergodicity of the chain and the expected chain. 

Theorem 7. (Infinite Flow Theorem) Let the random model {W(k)} be independent, and have 
a common steady state 7T > in expectation and weak feedback property. Then, the following 
conditions are equivalent: 

(a) The model is ergodic. 

(b ) The model has infinite flow property. 

(c) The expected model has infinite flow property. 

(d) The expected model is ergodic. 

Proof. First, we establish that parts (a), (b) and (c) are equivalent by showing that (a) ==>- (b) 
-<=>- (c) (a). In particular, by Theorem [l] we have § C J^, showing that (a) ==>- (b). By 

Theorem [3j parts (b) and (c) are equivalent. By Theorem [6j part (c) implies part (a). Now, we 
prove (a) -<=>- (d). Since (a) -<=>- (b), we have $ = & a.s. Hence, by Theorem [5J the parts (a) 
and (d) are equivalent. Q.E.D. 

The infinite flow theorem combined with the deterministic characterization of the infinite flow 
model of Theorem [3] leads us to the following result. 

Corollary 2. Let {A(k)} C § m be a deterministic model that has a common steady state vector 
7r > and weak feedback property. Then, the chain {A(k)} is ergodic if and only ifY!T=o^s(^) = 
oo for every nontrivial S C [m] . 

Under the conditions of Theorem [7j the model admits consensus, which follows directly from 
relation $ C ^ . 

Corollary 3. Let the assumptions of Theorem\2\hold. Then, the model admits consensus. 

The infinite flow theorem establishes the equivalence between the ergodicity of a chain and 
the expected chain for a class of independent random models. The central role in this result is 
played by the infinite flow and its equivalent deterministic characterization. Another crucial result 
is the interplay between the ergodicity and infinite flow of Theorem [4] yielding the equivalence 
between ergodicity of the chain and the expected chain. The following two examples are provided 
to illustrate some straightforward applications of the infinite flow theorem. 

i.i.d. Models. Consider an i.i.d. model {^(/c)}. Then, the expected matrix W = E[W(k)] is 
independent of k. Since W is stochastic, we have ir T W = tt t for a stochastic vector tt > 0. 
Therefore, an i.i.d. model is an independent model with a common steady state ir in expectation. 

In [3], it is shown that for the class of i.i.d. models that have a.s. positive diagonal entries, 
the ergodicity of the expected model and the ergodicity of the original model are equivalent. The 
application of this result is reliant on the condition of the a.s. positive diagonal entries, which 
implies that the model has feedback property, as shown in Corollary [TJ This property, however, 
is stronger than weak feedback property. At the same time, no requirement on the steady state 
vector is needed. 
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The application of the infinite flow theorem to the i.i.d. case would require weak feedback 
property and the existence of a steady state vector 7r > 0. Thus, there is a tradeoff in the 
conditions for the ergodicity provided by the infinite flow theorem and those given in [5]. To 
further illustrate the difference in the conditions, we consider the homogeneous deterministic model 
{A(k)} of Example |5} The model {A(k)} has weak feedback property and the steady state vector 
7r = |e, so the ergodicity of the model can be deduced from the infinite flow theorem. At the 
same time, as seen in Example [5j the model does not have positive diagonal entries and, therefore, 
the ergodicity of the model cannot be deduced from the results in [5j [JJ . In the light of this, the 
infinite flow theorem provides conditions for ergodicity that complement the conditions of [5j [7J . 
Gossip Algorithms on Time-varying Networks. As another application of the infinite flow theorem, 
we consider an extension of the standard gossip algorithm to time-varying networks. In particular, 
the gossip algorithm originally proposed in (36J [TT] is for static networks. Here, we give a sufficient 
condition for the convergence of a gossip algorithm for networks with time-changing topology. 
Consider a network of m agents viewed as nodes of a graph with the node set [m]. Suppose 
that each agent has a private scalar value Xj(0) at time k = 0. Now, let the interactions of 
the agents be random at nonnegative integer valued time instances k as follows: At any time 
k > 1, two different agents i,j G [m] wake up with probability Pij(k), where Pij{k) = Pji{k) 
and Yli<j Pij(k) = 1- Then, they set their values to the average of their current values, i.e., 
Xi{k) = Xj(k) = \{xi{k — 1) + Xj(k — 1)). The choices of the pairs {i, j} of interacting agents at 
different time instances are independent. 

Based on the agent interaction model, define the independent random model {H^(/c)} by: 

W{k) =1- ^(e, - e,)(e; - e 3 ) T with prob. P l3 {k). (21) 
Then, the dynamic system ([!]) driven by the random chain {H^/c)} describes the evolution of the 



vector x(k) that has its 2-th component value equal to agent i value, Xi(k). As seen from (21) 
any realization of the model {^(fc)} is doubly stochastic. Hence, the model has a common steady 
state ii = -j-e in expectation. Also, the model has strong feedback property (with 7 = |). 



Lemma 13. For extended gossip algorithm (21), the consensus is almost sure ifY2T=o p>s (^) = 00 
for any nontrivial set S C [m] . 

Proof. The extended gossip algorithm satisfies the assumption of the infinite flow Theorem [7j Since 
E[W tJ (k) + W^k)} = Pij(k) for all i ^ j and k > 0, it follows that EZo p s(k) = EZo E[Ws(*)]- 
Thus, by the infinite flow theorem the model admits consensus if YlT=o Ps(k) = 00 for any S C [m]. 
Q.E.D. 

When P(k) = P for all k as in [36, TT] . we can consider the graph G = ([m], E) where the edge 
{i, j} G E if and only if Py = Pji > 0. In this case, it can be seen that the condition of Lemma 13 



is equivalent to the requirement that the graph G is connected. One can further modify the 
algorithm in (21) to allow for time-varying weights, i.e., Xi{k) = a{k)xi{k — 1) + (1 — a{k))xj{k — 1) 



and Xj(k) = a(k)xj(k — 1) + (1 — a(k))xi(k — 1) with a(k) G (0, 1). Such a scheme is a natural 

6j. In this case, it can be verified that 



13 



still holds. 



generalization of the symmetric gossip model proposed in 
if a{k) G [a, 1 — a] for a G (0, |], then the result of Lemma 

6 Conclusion 

We have studied the ergodicity and consensus problem for a linear discrete-time dynamic model 
driven by random stochastic matrices. We have introduced a concept of the infinite flow event 
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and studied the relations among this event, ergodicity event and consensus event. The central 
result is the infinite flow theorem providing necessary and sufficient conditions for ergodicity of 
independent random models. The theorem captures the conditions ensuring the convergence of 
the random consensus algorithms, such as gossip and broadcast schemes [131 1121 EH]- Moreover, 
the infinite flow theorem captures simultaneously the conditions on the connectivity of the system 
and the sufficient information flow over time that have been important in studying the consensus 
and average consensus in deterministic settings [191 EQJ EH EHJ |29j [371 EE] • As illustrated briefly 
on two examples, the infinite flow theorem provides a convenient tool for studying the ergodicity 
of a model as well as consensus algorithms. Finally, we note that the work in this paper is readily 
extendible to the case when the initial state x(0) in is itself random and independent of the 
chain {W(k)}. 
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